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DETAILED ACTION 

Claims 1-31 are pending and have been examined. 

Specification 

1 . The disclosure is objected to because of the following informalities: 

• The following abbreviations should each be spelled out, with the abbreviation in 
parenthesis following it: 

o CPU (fourth paragraph of part MA on page 3) 

o PLAs, PALs, and ASICs (first paragraph on page 5) 

o LED (first paragraph on page 9) 

o GPS (last paragraph on page 16) 

o DVD (last paragraph on page 17) 

• "instantaneious" should be changed to "instantaneous" (second paragraph of part 
IV on page 7) 

Appropriate correction is required. 

2. The disclosure is objected to because it contains an embedded hyperlink (on 
page 13) and/or other form of browser-executable code. Applicant is required to delete 
the embedded hyperlink and/or other form of browser-executable code. See MPEP § 
608.01. 
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3. The use of the trademark ANOTO (on page 13) has been noted in this 
application. It should be capitalized wherever it appears and be accompanied by the 
generic terminology. 

Although the use of trademarks is permissible in patent applications, the 
proprietary nature of the marks should be respected and every effort made to prevent 
their use in any manner which might adversely affect their validity as trademarks. 

Claim Objections 

4. Claims 1-3, 7, 12, and 25-31 are objected to because of the following 
informalities: 

• As per claims 1-3, 7, 12, and 25-31, the bullets should be removed. 

• As per claims 1 , 25, and 28, the claims recite the limitation "said user's 
instantaneous position". There is insufficient antecedent basis for this limitation 
in the claims. It is suggested by the examiner that applicant amend the limitation 
to recite "a user's instantaneous input position". 

• As per claim 12, the semicolon after "wherein" should be removed. Additionally, 
the word "and" should be placed after the phrase "embedded in said medium 
itself;" on the same line as the phrase. 

• As per claims 25-27, "computer-readable medium" should be changed to 
"computer-readable storage medium" to conform to the specification. 
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• As per claim 29, the claim recites the limitation "said user's position". There is 
insufficient antecedent basis for this limitation in the claim. It is suggested by the 
examiner that applicant amend the limitation to recite "a user's input position". 
Appropriate correction is required. 

Claim Rejections - 35 USC § 112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claim 1 1 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

Claim 1 1 recites the limitation "said file index" in the first line of the claim. There 
is insufficient antecedent basis for this limitation in the claim. It is suggested by the 
examiner that applicant either change the dependency of the claim or amend the claim 
to recite The method of claim 7, wherein a file index was previously generated during 
creation of said tangible medium." 

Claims - 35 USC § 112, 6 th paragraph 

7. It is noted that Applicant has set forth in claim 28 a "means for receiving", "means 
for determining and accessing", and "means for enabling." 



Claim Rejections - 35 USC § 102 
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8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

9. Claims 1-31 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Zimmerman (U.S. Patent Application Publication 2002/0193975). 

Claim 1 

As per claim 1, Zimmerman teaches a method for assisting navigation of 
digital content using a tangible medium, comprising: receiving an instruction to 
access digital content corresponding to a portion of a tangible medium (in figure 1 

and in paragraph 20 on page 2, "The system generally comprises a probe 
device and a computer. The probe is manipulated over off-line 
media, preferably printed medium, and information is transmitted 
to the computer and matched to electronic media"), said medium being 
readable by a user-positionable input device (in figure 1 and in paragraph 20 on 
page 2, "The system generally comprises a probe device and a 
computer. The probe is manipulated over off-line media, 
preferably printed medium, and information is transmitted to the 
computer and matched to electronic media"), and said digital content 
being accessible from a stored file (in paragraph 118 on page 7, " All 
programming and data related thereto are stored in computer 
memory, static or dynamic, and may be retrieved by the user in 
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any of: conventional computer storage (local or remote), display 
(i.e., CRT) and/or hard copy (i.e., printed) formats"); determining 
and accessing stored digital content corresponding to said user's instantaneous 
position on said tangible medium (in paragraph 21 on page 2, "Based upon the 
position of the probe over the. off -line media, corresponding 
multimedia data is retrieved from storage and presented to the 
user"); and enabling electronic navigation of said digital content (in paragraph 21 
on page 2, "Preferably, the multimedia data and off-line media are 
designed so as to provide an educational experience in which, the 
synchronization of the probe position and output provides the 
visceral experience normally experienced by a user of a real 
tool. To simulate the real tool, the multimedia data corresponds 
to the output a real tool would produce when the real tool is 
used to inspect the items on the off-line media "). 

Claim 2 

As per claim 2, the rejection of claim 1 is incorporated and Zimmerman further 
teaches determining a change in position of said input device on said tangible 
medium; and obtaining a new stored file corresponding to said change in 
position (in paragraph 24 on page 2, "the printed medium additionally has 
a number of icons representing different tools to be simulated. 
By placing the probe over a particular icon , the tool 
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represented by the icon is selected such that the retrieved 
multimedia data corresponds to the selected tool "). 

Claim 3 

As per claim 3, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining digital signals representing a localized region of said tangible medium, 
said localized region being proximate to said position of said input device on said 
tangible medium (in paragraph 66 on page 4, "Referring to fig. 4, in a 
preferred embodiment of the present invention, probe 104 
contains a CMOS monochromatic camera 400 with wide angle lens 
406 and illumination system 410 to capture an image segment 408 
printed on the medium 106, underneath the probe . The image is 
detected and transmitted to the computer 100 by image processor 
402 , communicating to the computer 100 through connector 404, 
such as Universal Serial Bus (USB) the computer inherently obtains digital 
signals through a USB connector); determining at least one stored file 
corresponding to said localized region, and containing said digital content, by 
using pattern matching (in paragraph 66 on page 4, "The image segment 4 08 is 
a portion of a full image 412 appearing in the off-line media 
106. An image retrieval method running in the computer 100 
receives the image segment 408, and outputs image identification 
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and position information of the image segment 4 08" and in paragraph 68 
on page 4, "FIG. 5 shows a flow chart summarizing the methods of 
sampled image to reference image matching... At block 506 the 
salient features of the sampled image are extracted. At block 
508 the salient features of the sampled image are used to find 
the closest reference image " in view of figure 5; comparing salient features is 
one form of pattern matching); and retrieving an appropriate portion of said file to 
enable user navigation (in paragraph 21 on page 2, "Based upon the position 
of the probe over the off-line media, corresponding multimedia 
data is retrieved from storage and presented to the user... To 
simulate the real tool, the multimedia data corresponds to the 
output a real tool would produce when the real tool is used to 
inspect the items on the off-line media "). 

Claim 4 

As per claim 4, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern within 
said localized region with a pattern in said stored file (in paragraph 68 on page 4, 

"FIG. 5 shows a flow chart summarizing the methods of sampled 
image to reference image matching... At block 504 the sampled 
image is captured by the camera 400 in the probe 104. At block 
506 the salient features of the sampled image are extracted. At 
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block 508 the salient features of the sampled image are used to 
find the closest reference image " in view of figure 5). 

Claim 5 

As per claim 5, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern 
embedded within said medium itself (in paragraph 20 on page 2, "The system 
generally comprises a probe device and a computer. The probe is 
manipulated over off-line media, preferably printed medium , and 
information is transmitted to the computer and matched to 
electronic media" and in paragraph 45 on page 3, "Printed, off-line 
medium 106 has a suitable image 108 imprinted thereon"). 

Claim 6 

As per claim 6, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created independently of 
said file (in paragraph 87 on page 6, "In an application of the invention 
where the media 106 includes text, the user may circle text, or 
swipe the text with the probe 104 to select, translate, provide 
definition (e.g. from a dictionary), underline, highlight, make 
bold, copy or cut the corresponding electronic text appearing in 
the computer 100. In this application, images of the text, or 
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salient features of the text need not be stored in the computer 
100. In a preferred embodiment, text is stored as ASCII 
characters along with the font and other layout and contextual 
information to enable the computer to generate an accurate image 
of the page, or otherwise sufficiently represent the page"). 

Claim 7 

As per claim 7, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining coordinates of said position of said input device on said tangible 
medium, determining at least one stored file corresponding to said position and 
containing said digital content, determining coordinates within said stored file, 
corresponding to said input device position coordinates, by using coordinate 
mapping ("The computer decodes the bar code information into probe 
position and image identification , resulting in a synchronized 
image 204 on the computer screen of human bones 206" in view of figures 
2a and 2b; identifying a position on an image inherently involves obtaining and 
determining coordinates, and in this example, the probe position (coordinate) on the 
tangible medium is directly mapped to a position on the stored image); and using said 
determined coordinates to retrieve an appropriate portion of said file to enable 
user navigation ("... resulting in a synchronized image 204 on the 
computer screen of human bones 206" in view of figure 2b). 
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Claim 8 

As per claim 8, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said coordinate mapping involves a linear transformation from 
tangible medium coordinates to stored file coordinates (in paragraph 51 on page 3, 

"The computer decodes the bar code information into probe 
position and image identification, resulting in a synchronized 
image 2 04 on the computer screen of human bones 2 06" in view of figures 
2a and 2b; in this example, the probe position on the tangible medium is directly 
mapped to the position on a stored image, and therefore inherently involves a linear 
transformation from tangible medium coordinates to stored file coordinates). 

Claim 9 

As per claim 9, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein at least one of said tangible medium and said stored file includes a grid 
system (in paragraph 51 on page 3, "The computer decodes the bar code 
information into probe position and image identification, 
resulting in a synchronized image 2 04 on the computer screen of 
human bones 2 06" in view of figures 2a and 2b; identifying a position on an image 
inherently involves a coordinate or grid system). 
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Claim 10 

As per claim 10, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said determining said stored file includes utilizing a file index 
read from said tangible medium (in paragraph 49 on page 3, "fig. 2a 
illustrates the use of the present invention with position and 
image identification information encoded in bar codes printed on 
the off-line media "). 

Claim 11 

As per claim 1 1 , the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said file index was previously generated during creation of said 
tangible medium (in paragraph 49 on page 3, "fig. 2a illustrates the use 
of the present invention with position and image identification 
information encoded in bar codes printed on the off-line media "). 

Claim 12 

As per claim 12, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes a plurality of machine-readable 
patterns embedded in said medium itself (in paragraph 45 on page 3, "Printed, 
off-line medium 106 has a suitable image 108 imprinted thereon" 
and in paragraph 49 on page 3, "fig. 2a illustrates the use of the 
present invention with position and image identification 
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information encoded in bar codes printed on the off-line media "); 
said obtaining coordinates of said position of said input device is based on 
reading a unique pattern at said position of said input device, and analyzing said 
unique pattern to determine said coordinates (in paragraph 51 on page 3, "The 
probe 104 is placed over the picture of the hand 200 in book. 
The probe 104 sends bar code information to the computer. The 
computer decodes the bar code information into probe position 
and image identification , k resulting in a synchronized image 2 04 
on the computer screen of human bones 206" in view of figures 2a and 2b). 

Claim 13 

As per claim 13, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes an image, and said navigation 
includes displaying said image (in paragraph 81 on page 5, "The invention 
matches the sampled image 706 to a reference image of the frog's 
eye retrieved from storage device 102. Based on the probe 104 
position, reference image, and tool selected, a magnified view 
of a frog eye 802 is presented on the computer screen 800. The 
invention synchronizes the movement of the probe 104 to the 
display of the frog's eye 802. When the user moves the tool to 
the right, the image of the frog's eye on the computer 800 
screen would also move to the right, giving the user the 
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visceral experience of holding an actual microscope over a 
frog 1 s eye" in view of figure 7). 

Claim 14 

As per claim 14, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes audio, and said navigation includes 
playing said audio (in paragraph 77 on page 5, "when the user 616 touches a 
particular word, for example "forest 11 618 as show in FIG. 6, the 
computer 610 retrieving an audio clip of the spoken word 
"forest" from storage 614, and played it out speakers 62 0. When 
the user 616 touches the drawing 606, the sound of a bear 
growling occurs" in view of figure 6). 

Claim 15 

As per claim 14, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium serves as a video storyboard (in paragraph 

89 on page 6, "The remote server finds the best reference image 
match. In response to this match, the remote server sends a 
video clip 914 (for example MPEG1 compressed video) to the 
computer for presented on the computer display 912, in response 
to the sampled image in the magazine 904" in view of figure 9a and 9b). 
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Claim 16 

As per claim 16, the rejection of claim 1 is incorporated and Zimmerman further 
wherein said navigation includes at least one user-selectable mode (in paragraph 

83 on page 5, "Since the simulated display on the computer screen is 
not bound by the static reality of off-line media, the 
synchronized presentation may include dynamic images. For 
example when viewing the eye, the user can press a button on the 
probe 104 (not shown) to generate a virtual bright light at the 
probe, causing the image of the frog's eye 802 to retract in the 
socket"). 

Claim 17 

As per claim 17, the rejection of claim 16 is incorporated and Zimmerman further 
teaches wherein said modes are designated on, and selectable from, said tangible 
medium (in paragraph 24 on page 2, " the printed medium additionally has 
a number of icons representing different tools to be simulated. 
By placing the probe over a particular icon, the tool 
represented by the icon is selected such that the retrieved 
multimedia data corresponds to the selected tool " and in paragraph 66 
on page 4, "A presentation method running in the computer 100 
receives the image segment 408 information and presents 
multimedia content contained in a CD-ROM storage device 102 
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based on the virtual tool or function selected by the user " in view 
of figure 4). 

Claim 18 

As per claim 18, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes paper (in paragraph 21 on page 2, 

"The probe is manipulated over off-line media, preferably printed 
medium , " and in paragraph 26 on page 2, "Off-line media can be any non- 
networked media or object including books, magazines, 
newspapers, posters, pictures , mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, ■ 
plates, silverware, business cards , and clothing"). 

Claim 19 

As per claim 19, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes an optical device (in paragraph 2 on 

page 2, "The probe includes a camera to capture image segments and 
transmits them to a computing device"). 

Claim 20 

As per claim 20, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes a radio frequency device (in paragraph 
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23 on page 2, " The probe includes a camera to capture image segments 
and transmits them to a computing device , either local or 
remote, communicating through wired or wireless means " and in 
paragraph 45 on page 3, "... Probe device 104 is attached to computer 
100 via any suitable I/O interface so as to allow the probe to 
input data to computer 100. The I/O interface can including 
wired (e.g. USB, serial, parallel, firewire, optical 
communication) or wireless (e.g. 802.11, Bluetooth, UHF, 
infrared, CDMA, G3 , PCS, mobile phone, ISM band RF) means "). 

Claim 21 

As per claim 21 , the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium is two-dimensional, yet includes three- 
dimensional information (in paragraph 26 on page 2, "Off-line media can be 
any non-networked media or object including books, magazines, 
newspapers, posters, pictures , mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, 
plates, silverware, business cards, and clothing. . ."). 



Claim 22 

As per claim 22, the rejection of claim 1 is incorporated and Zimmerman further 
teaches using multiple tangible media to facilitate three-dimensional navigation (in 
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paragraph 72 on page 5, "When three sequential sampled images are 
identified as belonging to the same reference image, a match is 
declared", in paragraph 22 on page 7, "the multimedia data and off-line 
media is designed so as to provide a commerce experience in 
which the synchronization of the off-line media and electronic 
material enhances the knowledge of the user to the products in 
the off-line media, for example, showing different views or 
colors of a dress, or a simulation of the user in the dress "). 

Claim 23 

As per claim 23, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created using said stored 
file (in paragraph 50 on page 3, "the bar codes 202 are printed on clear 
plastic (e.g. mylar) sheets and adhered to the pages of the off- 
line media. This allows a book to be published using 
conventional printing techniques, and bar codes to be placed on 
the pages after conventional printing "; the bar codes refer to the stored 
files likely used in the printing of the book). 

Claim 24 

As per claim 24, the rejection of claim 1 is incorporated and Zimmerman further 
teaches implemented In a handheld portable electronic device ("the present 
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invention may be implemented on a conventional IBM PC or 
equivalent, mobile phone , personal digital assistant (PDA) , 
tablet computer , multi-nodal system (e.g., LAN) or networking 
system (e.g., Internet, WWW)"). 

Claim 25, 26, and 27 

Claims 25, 26, and 27 are the computer-readable medium claims corresponding 
to the method claims 1, 3 and 7 respectively, and are rejected under the same reasons 
set forth in connection with the rejection of claims 1 , 3, and 7. 

Zimmerman further teaches the computer-readable medium comprising logic 
instructions (in paragraph 118 on page 7, "The above system and its 
described functional elements are implemented in various 
computing environments... All programming and data related 
thereto are stored in computer memory , static or dynamic, and 
may be retrieved by the user. . ."). 

Claim 28 

Claim 28 is the system claim corresponding to the method claim 1 and is rejected 
under the same reasons set forth in connection with the rejection of claim 1 . 

Zimmerman further teaches the means for receiving (in paragraph 45 on page 

3, "Probe device 104 is attached to computer 100 via any suitable 
I/O interface so as to allow the probe to input data to computer 
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100. The I/O interface can including wired (e.g. USB, serial, 
parallel, firewire, optical communication) or wireless (e.g. 
802.11, Bluetooth, UHF, infrared, CDMA, G3 , PCS, mobile phone, 
ISM band RF) means" and in paragraph 1 18 on page 7, "For example, the 
present invention may be implemented on. . . mult i -nodal system 
(e.g., LAN) or networking system (e.g., Internet, WWW) "), means for 
determining and accessing (in paragraph 23 on page 2, "The computing device 
analyses the image segments, determines the image identity, and 
retrieves and presents electronic media corresponding to the 
image to the user"), and means for enabling (in paragraph 81 on page 5, "Based 
on the probe 104 position, reference image, and tool selected, a 
magnified view of a frog eye 802 is presented on the computer 
screen 800. The invention synchronizes the movement of the probe 
104 to the display of the f rog » s eye 802"). Therefore, Zimmerman 
teaches the system claimed. 

Claim 29, 30, and 31 

Claims 29, 30, and 31 are the system claims corresponding to the method claims 
1 , 3 and 7 respectively, and are rejected under the same reasons set forth in connection 
with the rejection of claims 1, 3, and 7. 

Zimmerman further teaches the interface (in paragraph 45 on page 3, "Probe 
device 104 is attached to computer 100 via any suitable I/O 
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interface so as to allow the probe to input data to computer. 
100. The I/O interface can including wired (e.g. USB, serial, 
parallel, firewire, optical communication) or wireless (e.g. 
802.11, Bluetooth, UHF, infrared, CDMA, G3 , PCS, mobile phone, 
ism band rf) means") and processor (in paragraph 23 on page 2, "The 
computing device analyses the image segments, determines the 
image identity, and retrieves and presents electronic media 
corresponding to the image to the user"; a computing device inherently 
comprises a processor). Therefore, Zimmerman teaches the system claimed. 



10. The prior art made of record and not relied upon is considered pertinent to 
applicants disclosure: 



Conclusion 



US 5133052 A 



Interactive graphical search and replace utility for 
computer-resident synthetic graphic image editors 



Bier; Eric A. et al. 



US 5583950 A 



Method and apparatus for flash correlation 



Prokoski; Francine J. 



US 5848373 A 



Computer aided map location system 



DeLorme; David M. et al. 



US 5933823 A 



Image database browsing and query using texture 
analysis 



Cullen; John et al. 



US 5991780 A 



Computer based system, method, and computer 
program product for selectively displaying patent text 
and images 



Rivette; Kevin Gerard et 
al. 



US 6081261 A 



Manual entry interactive paper and electronic 
document handling and processing system 



Wolff; Gregory J. et al. 
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US 6201176 B1 



US 6243492 B1 



US 6249590 B1 



US 6263335 B1 



System and method for querying a music database Yourlo; Zhenya 

Image feature extractor, an image feature analyzer 

and an image matching system Kamei; Toshio 

Method for automatically locating image pattern in 



digital images 

Information extraction system and method using 
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